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Positive Behavioral Interventions and Supports (PBIS) is an evidence-based approach 
to addressing behavior issues in schools. A significant amount of research has been done to 
identify the critical features of PBIS. More generally, implementation science points 
to a specific sequence to ensure that PBIS is implemented with fidelity. 


This spring status report provides information on PBIS implementation fidelity for school teams who are currently in 
PBIS training sponsored by the Minnesota Department of Education (MDE). The report summarizes average Tiered 
Fidelity Inventory (TFI) scores, the number of Self-Assessment Surveys (SAS) taken, and the types of data systems 
used by: 

=" 53 school teams that are in their second year of training (Cohort 13) 

= 47 school teams are in their first year of training (Cohort 14)! 


Tiered Fidelity Inventory 
The TFI measures implementation 
The PBIS State Leadership Team expects that school teams in their first year 
of training (Cohort 14) will score at least 50% on their Tier 1 TFI at spring 
training. As it is their second year of training, Cohort 13 school teams are 
expected to meet the 70% implementation fidelity benchmark at spring 
training. Some key findings from the spring 2019 TFI assessments include: 


fidelity for each of the three tiers of 
PBIS. It is designed to be taken four 
times per year while a school team is 
in training. In Minnesota, schools 
complete a TFI at each PBIS training 


=" The overall average Tier | TFI scores for Cohorts 14 increased and the and an additional TFl assessment is 
overall average for Cohort 13 decreased. The overall average for Cohort completed at their school with an 
14 increased by 14 percentage points since Winter 2018 (Figure 1). external evaluator. The external 

= The average score for all teams in each region was 72%, which exceeded evaluator visits their partner school 
the 70% benchmark. On average, school teams in Cohort 13 from the and completes a “walk-through” to 
North and South regions are implementing PBIS with fidelity according _ interview staff and students in 
to their TFI scores, while school teams from the Metro fell just below addition to completing the TFI 
this threshold. facilitation session. Schools in 


"Cohort 14 school teams from each region, on average, exceeded the training are currently only taking the 


winter training benchmark (60%) (Figure 2). Wier [eden Gigs VARESE SS ULI: 
In order to be implementing PBIS 


with fidelity, a school needs to score 
70% or better for each of the three 
tiers of PBIS. 


There were smaller numbers of school teams from the North region (N=9) in Cohort 13 and the South region (N=8) in 
Cohort 14. Caution should be used when interpreting results from these regions. 


1. Average Tier 1 TFl scores by region—Cohort 13 (Fall 2017 through Spring/End of Year 2019) 
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*Total is the average across all regions for all completed TFls 


2. Average Tier 1 TFl scores by region—Cohort 14 (Fall 2018-Spring 2019) 


eae 56% 53% 56% 
43% 42% 39% 42% 
: = Fall 2018 

a ‘ al a - = Winter 2018 
= Spring 2019 

a — a bee average 

(N=20, 19,15) (N=18,17,13) (N=8, 8, 6) across all 
regions 
(N=46, 44, 34) 
Self-Assessment Survey 


For schools in their first year of training (Cohort 14), the SAS should be 


completed once in the fall and once in the spring. For schools in their second The Self-Assessment Survey, or 
year of training (Cohort 13), school teams need to complete a SAS once at the SAS, examines the status and need 
end of the year. Some key findings include: for improvement of four behavior 


: support systems: school-wide, non- 
=" Nearly three-quarters (73%) of school teams in Cohort 13 have 


completed a SAS as of June 2019 (Figure 3). However, 27% of school 
teams in Cohort 13 did not complete a SAS during the 2018-2019 school 
year. The North region had the highest percentage of SAS completion 
(90%). 

=" Eighty-eight percent of school teams from Cohort 14 have completed at 
least one SAS assessment during the 2018-2019 school year. Of these, 
62% completed two or more SAS assessments. However, 26% of school 
teams in the North region did not complete a SAS this school year. All 
school teams from the South region have completed a SAS. 


classroom, classroom, and individual 
students. A summary of the survey 
results is used to develop an action 
plan for implementing and sustaining 
PBIS systems throughout the school. 
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3. SAS completed by Cohorts 13 and 14 during the 2018-2019 school year (by region) 
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Behavioral data system 
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At 2019 spring training, school teams were asked to identify which ODR data 
system(s) they use to track data. Schools are expected to have an established 
behavior system in place by winter training in their first year. Some key findings 


The presence of a behavioral data 
system allows for schools to track 


include: 


=" Sixty percent Metro schools and half of schools from the North region 
in Cohort 13 report they use SWIS or SWIS and another data system 
(Figure 4). Very few schools from the South region (4%) report they 


had no data system in place. 


=" Nine in ten school teams from all regions in Cohort 14 have a data 
system in place. Higher proportions of schools in the Metro and South 
regions report using a data system other than SWIS. Eleven percent of 
school teams from the North region report they do not have a data 


system in place. 


4. School behavioral data systems, Cohorts 13 & 14 
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office discipline referral (ODR) data 
and use this for data-based decision- 
making. ODR data includes whether 
the problem behavior, locations, 
possible motivations, others involved, 
and administrative consequences. 
The data collected in these systems 
are used for improving school-wide 
behavior support. 
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Sharing key data with school staff 


At spring training, Cohort 13 and 14 school teams were asked about how often they share implementation fidelity and 
outcome (ODR) data with school staff. School teams from Cohort 14 were asked about their level of confidence in 
sharing these types of data with other school staff. School teams from Cohort 13 were also asked if they disaggregate 
data by gender, disability or special education status, and/or race/ethnicity. Key findings include: 


=" The majority of Cohort 13 teams reported sharing implementation fidelity and outcomes data at least once a 
year, though some schools from the Metro and North regions indicated they have not shared or do not share 
this data with school staff (Figure 5). Higher proportions of school teams from the North region in Cohort 14 
report they have not shared or do not share implementation fidelity data. Conversely, 58% of school teams 
from the North region in Cohort 14 report they share their outcome data “every other month or more.” 

"Most school teams in Cohort 14 reported that they felt confident in sharing both types of data with other 
school staff. School teams from the North and South regions in Cohort 14 were more confident in sharing 
outcome data than sharing implementation data with other school staff (Figure 6). 

=" Higher proportions of school teams from the Metro region reported that they disaggregate data by race/ 
ethnicity, gender, and by disability or special education status compared with school teams from the North 
and South regions (Figure 7). 


5. Frequency of implementation and outcome data sharing with other school staff (Cohort 13 & 14) 


Cohort 13 
Metro North South 
(n=11) (n=10) (n=20) 


Cohort 14 
Metro North South 
(n=18) (n=12) (n=6)4 


Implementation fidelity data sharing 
Every other month, or more often? 
Two to five times a year> 
Once a year 27% 

Have not shared/Do not share® 


10% 
30% 


0% 0% 
17% 
17% 


Outcome data sharing 
Every other month, or more often? 
Two to five times a year® 27% 
Once a year 18% | 0% 25% 
Have not shared/Do not share*® 18% | 10% 5% 
Note. Percentages in table may not equal 100% due to rounding. 
4 Includes "Once a month or more" and "Every other month" response options 


11% | 0% | 0% 


28% | 33% | 50% — 


>: Includes "Three to five times a year" and "Twice a year" response options 
*: Includes "Have not shared" and "Does not share" response options 
4: Due to the small number of schools in the South region who answered this survey question, please use caution when interpreting and comparing results. 
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Schools from Cohort 14 were asked to rate their degree of confidence regarding sharing implementation fidelity and 
outcome data with other school staff (Figure 6). Most schools from each of the regions report they are confident in 
sharing implementation fidelity data with other school staff. However, 22% of school teams from the Metro region 
report they are not too confident to share this data. While all schools from the South region indicated they were 
confident in sharing outcome data with other school staff, some school teams from the Metro and North regions 
reported lower levels of confidence with regard to sharing this data. Eleven percent of school teams from the Metro 
region were not at all confident in sharing ODR data with other school staff. 


6. Confidence level of sharing implementation fidelity and outcome data with other school staff—Cohort 14 (N=36) 
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= Not too confident = Not at all confident = Not too confident = Not at all confident 


Cohort 13 schools were asked to identify whether or not they disaggregate ODR data by race/ethnicity, gender, or 
disability or special education status (Figure 7). School teams from the Metro region were more likely to indicate that 
they disaggregate outcome data by race/ethnicity, gender, or disability or special education status compared with the 
North and South regions. Eighty percent of school teams from the North region and two-thirds of school teams from 
the South region report they do not disaggregate data by race. 

7. ODR data disaggregation by region—Cohort 13 (N=41) 
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Issues to consider 


Based on the summary results described above, there are a few things the Minnesota PBIS Statewide Leadership Team 
(SLT), including the Minnesota Department of Education (MDE) and the Regional Implementation Partners (RIPs) 
should consider when planning future PBIS trainings and working with school teams. 


The SLT and the RIPs should This year, the SLT, in an attempt to garner a higher response rate of school 
continue to support of schools _ teams participating in the external TFI evaluation and walk-through, changed 


in training by encouraging the requirements of when a school team had to complete a TFI. School teams, 
them to complete and use their instead of taking a TFI at Spring 2019 training, were set up with a partner 
TFI and SAS data. school and encouraged to do the TFI and the TFI walk-through on their own 


instead. In previous years, school teams completed one TFI assessment at 
spring training and another one with an external evaluator. However, fewer 
school teams completed a TFI in the Spring of 2019 compared to the previous 
times during the school year. As a result, the SLT decided they would require 
all TFI assessments will take place during training and the walk-through 
will be optional for school teams. Of note, overall average TFI scores also 
decreased by one percentage point from Winter 2018 training to Spring 2019 
training for Cohort 13—this could be due to the small number of schools 
completing a TFI. The average TFI score for Cohort 14 rose from 42% to 
56%, which is above the benchmark for Spring training (50%). 


Twenty-seven percent of school teams from Cohort 13 also did not complete a 
SAS. The majority of Cohort 14 school teams have completed a SAS. The SLT 
and the RIPs should continue to tailor their supports to school teams in order to 
be sure they are getting the most out of training, working to improve TFI scores, 
and are in line with MDE’s assessment calendar to complete a TFI and SAS. 


Encourage school teams Most school teams have a data system that collects outcome (ODR) data. While 
to share implementation most Cohort 13 school teams report sharing implementation fidelity at least 
fidelity data more often using once a year, some school teams in all regions report that they have not shared 
creative approaches. or do not share this data. Also, Cohort 14 school teams from the Metro and 


North region were more likely to be confident in sharing outcome data 
compared with implementation fidelity data. Of note, Cohort 14 school 
teams noted that while they felt confident sharing this data, over one-quarter 
of teams from the Metro and one-third from the North report that they “have 
not shared or do not share this data.” Similar proportions of school teams 
said the same of outcome data. This illustrates a gap with data sharing— 
schools note they feel confident to share this data, but many of them do not. 
The RIPs and trainers should continue to provide some technical assistance 
around sharing these types of data with other school staff. There are a variety 
of interactive data sharing techniques that would align well with PBIS, such 
as hosting a data party and creating an interactive data placemat so school 
staff can understand and buy in to using their data. 
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Issues to consider (continued) 


Educate school teams around 
the importance of data 
disaggregation and explore 
reasons why school teams 
choose not to do so. 


Wilder 
Research 


Information. Insight. Impact. 


451 Lexington Parkway North 
Saint Paul, Minnesota 55104 
651-280-2700 
wilderresearch.org 


School teams from the Metro region are more likely to report that they 
disaggregate data by race/ethnicity, gender, and disability or special education 
status. However, there are still a few school teams in this region that do not 
disaggregate this data. School teams from all regions were more likely to 
report they disaggregate data by gender compared to race/ethnicity or 
disability or special education status. Additionally, school teams from the 
North region were less likely to report they disaggregate their data. The 
RIPs and trainers should continue to emphasize the importance of data 
disaggregation and work with schools to do this using SWIS or their data 
system so teachers and staff can more accurately address behavioral issues 
and other issues such as implicit bias or lack of community involvement with 
a school’s PBIS initiative. 


For more information 


This summary presents highlights of the spring 2019 Regional PBIS 
Trainings in Minnesota, which are sponsored by the Minnesota Department 
of Education. For more information, contact Nicole MartinRogers at Wilder 
Research, 651-280-2682. 
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